1. Field of the Invention
The present invention relates to image processing apparatuses and image processing methods, and in particular those used in saving image data obtained by capture with an imaging device such as a digital camera or the like.
2. Description of the Related Art
Currently, centering on single-lens reflex cameras, it is usual for the following two types of data to be selectable in digital cameras when recording in order to save image data obtained by imaging. That is, JPEG data as general-purpose image data in which general image processes (hereinafter referred to as “development processing”) for use as an image for viewing and compression processes have all been carried out, and raw data from the imaging device, which is generally referred to as raw data.
The former, JPEG data, has the following advantages. Since the compression ratio of image data is higher for JPEG data compared to raw data, it involves saving a smaller amount of data. Consequently, a large number of JPEG images can be saved on recording media such as semiconductor memory cards. Furthermore, JPEG data is highly usable data that can be decoded on many platforms including commonly used PCs (personal computers) and digital televisions and can be displayed for viewing.
On the other hand, despite the large data size, raw data has the following advantages. Namely, raw data is image data in which the raw data from the imaging device is faithfully saved. Consequently, it is possible to record and save the tones (bit depth) of image data obtained directly from the imaging device as they are. For example, with JPEG data, the image data from the imaging device that has been quantized at 12 bits undergoes tone compression down to 8 bits, but raw data is saved and recorded as it is at 12 bits.
When simply subjecting raw data to electrical development processing for display on a display device, the difference with JPEG data is difficult to judge. However, when printing with raw data, a printing result having excellent tonality and a high color range can be achieved by performing development processing that matches the color range of the printer to be used. Moreover, there is little effect of image deterioration when image processing is performed and it has strong durability as image information. Thus it has a major advantage in that it is unlikely for image processing to cause tone jump or tonal loss.
Depending on the digital camera maker, specialized software is commercially available that enables fundamentally the same development processing of a digital camera to be achieved on a PC. By using such specialized software, the various types of parameters for development (hereinafter referred to as “development parameters”) that are set in a digital camera at the time of capture can be reset on a PC to perform development processing on the raw data. Consequently, even when incorrect development parameters are used at the time of capture for example, if capture is performed using a raw data format, it is possible to perform development processing on a PC using appropriate development parameters. For this reason, raw data formats are currently widely used particularly among professional photographers who intend to use the image data, which they have obtained by capture, by processing it on a PC.
However, ordinary platforms (such as PCs and digital televisions) do not have a means for viewing raw data as an image. In this sense it can be said that raw data has poor usability. Consequently, ordinarily a user will install on a PC specialized application software for raw data and will only be able to view the raw data after the specialized application has executed development processing on the raw data. Thus there is a problem for the user in that it takes time and effort to view raw data.
If development processing functions were provided in each OS and each type of retouching software and the like, the time and effort of installing a specialized application could be eliminated. However, to obtain a development processing result having high picture quality, it would be necessary to develop development processing functions for each model of digital camera (or each model group having similar characteristics). This would result in development costs increasing undesirably.
To combat this problem, Japanese Patent Laid-Open No. 2004-304712 divides development processing functions into a portion (model-specific development processing portion) in which processing specific to the model (or model group) is carried out, and a portion (general purpose development processing portion) in which processing that is not dependent on the model is carried out. Also a configuration is disclosed in which a processing result of the model-specific development processing portion is outputted as intermediate format image data (hereinafter referred to as “intermediate raw data”) and this result is inputted to the general purpose development processing portion.
With this configuration, if developers of each OS and retouching software on PCs first develop a general purpose development processing portion, then it is possible achieve high picture quality development processing for each digital camera by developing only the model-specific development processing portion. Or it would be possible for each digital camera maker to provide only the model-specific development processing portion, which would be incorporated into each OS and retouching software. In this way, image processing more specific to each digital camera is carried out using the model-specific development processing portion and an image processing result having high picture quality can be obtained by processing this result using the general purpose development processing portion.
By configuring in this manner, the user is able to use only the general purpose imaging processing application that he/she is used to using, and high picture quality development processing can be carried out on raw data from digital cameras of various models of various makers.
Data processing of raw data on a PC involves the user changing the development parameters in image processing application software (hereinafter referred to as “development processing software”) that is capable of development processing. For example, when the contrast in image data obtained by capture is insufficient, the user increases the contrast value at the user interface (hereinafter referred to as “UI”) of the development processing software. Upon doing this, development processing is carried out on the raw data by the development processing software using the contrast parameters that have been updated, the result of which is displayed on the UI. Then, when a satisfactory development processing result is achieved, it is common for the development processing result to be savable as general purpose image data such as TIFF and JPEG or the like. It should be noted that when first displaying raw data, which has been arbitrarily obtained by capture, with development processing software, it is common that the result is displayed by carrying out development processing using the development parameters that have been set on the digital camera side.
Development parameters capable of being set in this manner vary more or less depending on the development processing software, but as one example the following parameters can be set.
Parameters are listed below as [Development parameter item name: selectable item value]
[White balance: auto, day light, cloud, tungsten, fluorescent, strobe]
[Tone curve: such as indicating points on a spline curve on X and Y axes]
[Contrast: −5 to +5] (0 is default, stronger contrast for greater values)
[Sharpness: −5 to +5] (0 is default, edges more strongly emphasized for greater values)
[Color density: −5 to +5] (0 is default, higher saturation for greater values)
When anticipating these parameters, the details to be processed for items such as the tone curve, contrast, sharpness, and color density are comparatively definite, and they are processing details that have little dependency on each model and can be made readily adapted for multiple uses. Furthermore, in terms of the processing details, even when these processing details are performed as post-processing at a stage after development processing on RGB data for viewing, these processing details have little effect on picture quality as long as the bit depth can be maintained. Image processing details such as these that have little dependency on model are the processing details to be processed by the general purpose development processing portion provided in the configuration disclosed Japanese Patent Laid-Open No. 2004-304712.
On the other hand, in regard to white balance processing, highly precise white balance adjustments can be carried out using characteristics of each model, and the precision of white balance is an important factor for the performance of a digital camera. Accordingly, development processing results having higher picture quality can be obtained by performing processing in the model-specific development processing portion, in which a large number of the results of algorithm research and development by various digital camera makers is incorporated and that is provided by the various digital camera makers.
Although the absorption of a wide range of model-specific characteristics and the incorporation of very good algorithms is implemented from various digital camera makers, a basic flow of white balance adjustment processing is as shown in an example below.
First, an optical black value is subtracted from the raw data as required. Next, when there is unevenness in color gain in each individual digital camera, an adjustment value is referenced to normalize the unevenness in color gain.
Next, the raw data is divided into a plurality of blocks having a predetermined width and height, and a color evaluation value is calculated for each block as shown in expression 1 below. It should be noted that expression 1 below is an example of a conversion formula in a case of primary color signals.Cx=(R−B)/Y Cy=(R+B−2G)/Y Y=(R+G+B)/2  expression 1
Here, when color evaluation values (Cx, Cy) are contained in a “white detection area that is preset for each light source”, that block is assumed to be white. Then, integration values of each color pixel (SumR, SumG, and SumB) are calculated respectively for each block assumed to be white. From the thus-calculated integration values, white balance gain (white balance coefficient values) kWB_R, kWB_G, and kWB_B are calculated using expression 2 shown below.kWB—R=k×(SumR+SumG+SumB)/SumR kWB—G=k×(SumR+SumG+SumB)/SumG kWB—B=k×(SumR+SumG+SumB)/SumB  expression 2
Here, k may be set such that overall gain becomes a preferable value.
It should be noted that “white detection area that is preset for each light source” refers to an area that is determined for each light source such as day light, fluorescent, tungsten and the like, and is calculated from one or a range of a plurality of color temperatures.
The range of color temperatures predetermined for each light source involves details of algorithm development strived for by each digital camera maker for the best values and involves details for tuning development processing parameters. Furthermore, even when a digital camera is set to “day light” for example, the color temperature of the light source varies due to conditions such as time, direction, and weather as shown below.
Sky light: 18,000 to 12,000 K (K=Kelvin)
North side sky in fine weather: 12,000 K
Noon sunlight: 5,300 K
Sunrise or sunset: 3,500 to 2,700 K
In other words, when the light source is set to “day light”, white detection is carried out in regard to a plurality of white detection areas calculated from the above plurality of color temperature ranges, and after obtaining the most probable color temperature from among the above, it is necessary to calculate SumR, SumG, and SumB for the corresponding white detection areas.
FIG. 1 shows an example of a white detection area when there is a setting of “light source=auto”. When there is an auto setting, it is necessary to perform white detection ranging across all color temperatures. For this reason, white is captured at each arbitrary color temperature step under light sources ranging from a high color temperature to a low color temperature, then color evaluation values are calculated from signal values obtained from the imaging device, and these are plotted. In this way, white determination lines (white detection axes) are generated from a high color temperature to a low color temperature as shown in FIG. 1. In fact, since there is slight unevenness in light separation even for white, a range enclosed by the dotted lines having a slight width from the obtained white determination line is used as the white detection area at times of the auto setting. At times when the light source is other than the auto setting, a corresponding range of one or a plurality of color temperatures within FIG. 1 is set as the white detection area.
When carrying out highly precise white balance adjustments with the example of the above white balance coefficient value calculation method, rather than using white balance coefficient values that are predetermined for each light source, it is necessary to obtain white balance coefficient values from a result of referencing the content of the image data.
Furthermore, for white balance, it is necessary to obtain a white detection area from observed values for each model (or model group) of digital camera. Consequently, each digital camera maker sets the performance thereof as an important feature of the digital camera, and is advancing research and development in algorithms and tuning parameters. Also in this sense, to carry out very highly precise white balance adjustments, it is preferable that the white balance coefficient value calculation processing is performed by the “model-specific development processing portion” in the configuration of the aforementioned Japanese Patent Laid-Open No. 2004-304712.
However, for image data in which white balance (hereinafter referred to as “WB”) processing has been performed by the model-specific development processing portion and outputted as intermediate raw data, it is desirable to readjust WB using the general purpose development processing portion even when highly precise WB adjustment has already been performed.
For example, reasons for this include:    1) Capturing was performed using the auto setting and development processing has been performed, but a color cast has occurred, and therefore it is desired to reset to the actual light source at the time of capture. That is, although the precision in the latest WB processing has improved and it is possible to achieve appropriate WB adjustments for many types of image data, in cases such as where there is absolutely no white areas in the image data, there is a possibility that a color cast will occur in the processing result for image data under the auto setting. In cases such as where capture was performed with the light source setting of the digital camera set to the auto setting and color a cast has occurred in the result of processing in the model-specific development processing portion with the auto setting, a demand arises for WB readjustments using the general purpose development processing portion.
2) Capturing was performed using the auto setting and development processing has been performed, but it is desired to retain a certain atmosphere and therefore it is desired to reset to day light. For example, in cases such as where capture was performed using a candle flame as a light source, there are times when the WB adjustment result under the auto setting is insufficient for the atmosphere of the photo. In cases such as where it is desired to retain the candlelight atmosphere and reproduce the photo with coloring in which white subjects are shifted to warmer color types, it becomes necessary to readjust WB once again under a setting of day light or the like.
3) There was a mistake in the light source setting at the time of capture and therefore it is desired to reset to the actual light source at the time of capture. For example, there are cases where the user undesirably carries out capture by inadvertently leaving the setting on the digital camera to a light source that is different from the actual light source. In such cases, it becomes necessary to reset the light source to that of the time of capture for WB result that was automatically processed by the model-specific development processing portion, then once again carry out WB readjustments using the general purpose development processing portion.
However, the WB algorithms and tuning parameters of the model-specific development processing portion do not conventionally reside in the general purpose development processing portion. For this reason, conventional techniques have been unable to carry out WB readjustment processes making use of the algorithms and tuning parameters of the model-specific development processing portion.